Add source-aware acceptance eval fixtures - #80
Closed
catoncat wants to merge 1 commit into
Closed
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This was referenced Jun 26, 2026
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #73
Scope
Add the source-aware acceptance-eval slice for the experimental public source adapter hardening work.
This PR extends the synthetic acceptance gate so it covers:
find->read-rangefor message-level evidence.find->read-pagefor page context.Implementation summary
eval/acceptance-gate.tsnow writes a fixture bundle with per-source roots forcodex,claude-code, andpi.runAcceptanceGate()syncs all three roots into the same temporary DB and returnssourceRootsplussourceSyncs, while preserving the existingfixtureRootandsyncCodex fields.claude-code-message-range-contextpi-session-page-contexteval/acceptance-gate.test.tsnow asserts Codex + Claude Code + Pi sync counts and 6/6 hard-case pass coverage.Verification
npx vitest run eval/acceptance-gate.test.ts-> 1 file passed, 1 test passed.npm run eval:acceptance-> 6 hard cases, 6 pass, 0 fail; Codex added 6, Claude Code added 1, Pi added 1.npm run check->tsc --noEmitplus Vitest, 31 files passed, 206 tests passed.git diff --check -- eval/acceptance-gate.ts eval/acceptance-gate.test.ts-> passed.Risk
Low. This changes eval/test code only and does not modify production parser projection, source adapter acceptance policy, SQLite schema, or CLI behavior.
Base branch / dependencies
Base branch:
agent/issue-71-query-profileDepends on:
Related but not blocking:
Suggested merge order:
Trace
request_id:
cxs_continue_roadmap_to_completion_20260626T054330Z_api_appendSummary by cubic
Make the acceptance eval source-aware by writing per-source fixtures and syncing
codex,claude-code, andpiinto one DB. Adds hard goldens for Claude Code message-range and Pi page-context to meet #73.sourceRootsandsourceSyncswhile keepingfixtureRoot/syncforcodex.claude-code-message-range-contextandpi-session-page-context.findselectors with context modes:claude-codeusesread-range,piusesread-page.Written for commit 1c97c73. Summary will update on new commits.